Conversation
8eacbb6 to
f4a755d
Compare
Polar Signals Profiling ResultsLatest Run
Previous Runs (5)
Powered by Polar Signals Cloud |
Benchmarks: PolarSignals ProfilingVortex (geomean): 0.925x ➖ datafusion / vortex-file-compressed (0.925x ➖, 1↑ 0↓)
|
Benchmarks: TPC-H SF=1 on NVMEVerdict: No clear signal (environment too noisy confidence) datafusion / vortex-file-compressed (0.983x ➖, 1↑ 0↓)
datafusion / vortex-compact (0.979x ➖, 0↑ 0↓)
datafusion / parquet (0.978x ➖, 2↑ 1↓)
datafusion / arrow (0.968x ➖, 2↑ 0↓)
duckdb / vortex-file-compressed (0.984x ➖, 0↑ 0↓)
duckdb / vortex-compact (0.977x ➖, 0↑ 0↓)
duckdb / parquet (0.991x ➖, 3↑ 4↓)
duckdb / duckdb (0.984x ➖, 1↑ 0↓)
Full attributed analysis
|
Benchmarks: FineWeb NVMeVerdict: No clear signal (low confidence) datafusion / vortex-file-compressed (0.968x ➖, 0↑ 0↓)
datafusion / vortex-compact (0.963x ➖, 1↑ 0↓)
datafusion / parquet (0.986x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed (1.001x ➖, 1↑ 2↓)
duckdb / vortex-compact (1.009x ➖, 0↑ 1↓)
duckdb / parquet (0.932x ➖, 3↑ 0↓)
Full attributed analysis
|
f4a755d to
9f41fc3
Compare
Merging this PR will degrade performance by 17.72%
Performance Changes
Comparing Footnotes
|
Benchmarks: TPC-DS SF=1 on NVMEVerdict: No clear signal (low confidence) datafusion / vortex-file-compressed (0.891x ✅, 57↑ 0↓)
datafusion / vortex-compact (0.897x ✅, 57↑ 0↓)
datafusion / parquet (0.908x ➖, 41↑ 0↓)
duckdb / vortex-file-compressed (0.904x ➖, 44↑ 0↓)
duckdb / vortex-compact (0.926x ➖, 22↑ 0↓)
duckdb / parquet (0.930x ➖, 14↑ 0↓)
duckdb / duckdb (0.929x ➖, 24↑ 0↓)
Full attributed analysis
|
Benchmarks: Random AccessVortex (geomean): 0.915x ➖ unknown / unknown (0.967x ➖, 6↑ 0↓)
|
🚨🚨🚨❌❌❌ SQL BENCHMARK FAILED ❌❌❌🚨🚨🚨Benchmark |
Benchmarks: TPC-H SF=1 on S3Verdict: No clear signal (environment too noisy confidence) datafusion / vortex-file-compressed (0.997x ➖, 1↑ 0↓)
datafusion / vortex-compact (1.136x ➖, 0↑ 3↓)
datafusion / parquet (1.023x ➖, 1↑ 4↓)
duckdb / vortex-file-compressed (1.075x ➖, 0↑ 0↓)
duckdb / vortex-compact (1.075x ➖, 0↑ 0↓)
duckdb / parquet (1.034x ➖, 0↑ 0↓)
Full attributed analysis
|
Benchmarks: FineWeb S3Verdict: No clear signal (environment too noisy confidence) datafusion / vortex-file-compressed (0.939x ➖, 1↑ 0↓)
datafusion / vortex-compact (1.025x ➖, 0↑ 0↓)
datafusion / parquet (1.152x ➖, 0↑ 2↓)
duckdb / vortex-file-compressed (1.092x ➖, 0↑ 0↓)
duckdb / vortex-compact (1.025x ➖, 0↑ 0↓)
duckdb / parquet (1.018x ➖, 0↑ 0↓)
Full attributed analysis
|
Benchmarks: Statistical and Population GeneticsVerdict: No clear signal (low confidence) duckdb / vortex-file-compressed (1.038x ➖, 0↑ 1↓)
duckdb / vortex-compact (1.030x ➖, 0↑ 1↓)
duckdb / parquet (1.011x ➖, 0↑ 0↓)
Full attributed analysis
|
🚨🚨🚨❌❌❌ SQL BENCHMARK FAILED ❌❌❌🚨🚨🚨Benchmark |
🚨🚨🚨❌❌❌ SQL BENCHMARK FAILED ❌❌❌🚨🚨🚨Benchmark |
Signed-off-by: Adam Gutglick <adam@spiraldb.com>
Signed-off-by: Adam Gutglick <adam@spiraldb.com>
b9f24ac to
a7e8958
Compare
File Sizes: PolarSignals ProfilingNo file size changes detected. |
File Sizes: TPC-H SF=1 on NVMEFile Size Changes (220 files changed, -98.5% overall, 0↑ 220↓)
Totals:
|
File Sizes: FineWeb NVMeNo file size changes detected. |
File Sizes: TPC-DS SF=1 on NVMENo file size changes detected. |
File Sizes: Statistical and Population GeneticsNo file size changes detected. |
Benchmarks: CompressionVortex (geomean): 0.999x ➖ unknown / unknown (1.002x ➖, 1↑ 2↓)
|
The main idea is to only build the benchmark binaries once, instead of doing it for each one. It both saves money and unifies the cache between all the builds. The binary is then passed through the artifact cache to the machines that end up running the benchmarks.